VL-LTR: Learning Class-wise Visual-Linguistic Representation for Long-Tailed Visual Recognition

Recently, computer vision foundation models such as CLIP and ALI-GN, have shown impressive generalization capabilities on various downstream tasks. But their abilities to deal with the long-tailed data still remain be proved. In this work, we present a novel framework based pre-trained visual-linguistic for recognition (LTR), termed VL-LTR, conduct empirical studies benefits of introducing text modality Compared existing approaches, proposed VL-LTR has following merits. (1) Our method can not only learn visual representation from images but also corresponding linguistic noisy class-level descriptions collected Internet; (2) effectively use learned improve performance, especially classes fewer image samples. We extensive experiments set new state-of-the-art performance widely-used LTR benchmarks. Notably, our achieves 77.2% overall accuracy ImageNet-LT, which significantly outperforms previous best by over 17 points, is close prevailing training full ImageNet. Code available at https://github.com/ChangyaoTian/VL-LTR .